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(57) Abstract 

Raw digital video data is generated by a digital video camera and transmitted over a bus (27) to a computer (28). The raw digital 
video data is generated from the analog output of a charge coupled device (23) in the camera, which preferably includes color filter overlays. 
The bus can be a relatively low-speed bus, having a bandwidth of not greater than 12 Mbits/second. The video camera preferably processes 
the raw digital video data prior to transmitting it over the bus to decrease the resolution of the image defined by the raw digital video 
data. Such processing can be downsampling in both the horizontal and vertical dimensions. The computer includes a processor capable of 
converting the raw digital video data into usable digital video data, such as YUV or YIQ digital video data. 
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COLOUR VIDEO DATA REDUCTION 

BACKGROUND OF THE INVENTION 

1. Field of the Invention 

This invention relates to the field of video color format and compression. 

2. Description of the Related Art 

The advent of powerful microprocessors has expanded the quantity and quality of 
computer applications. Multimedia and other video related applications in particular are 
increasingly prominent. These applications typically involve transmitting video data into and out 
of personal computers (PCS). In typical video related applications, as shown in Fig. 1, a digital 
video camera 10 is used to capture video 18. Digital video camera 10 typically includes a lens 
12, a single plate charge coupled device (CCD) sensor 13, correlated double sampling and 
automatic gain control device 14, analog to digital (A/D) converter 1 5, digital signal processor 
(DSP) 1 6 and microcontroller 1 7. 

As noted, typical video cameras for PC and home applications use a single plate CCD. 
Therefore, to produce a color video signal, the CCD's are covered with a thin film color filter, 
which can have various color formats. For example, there are filters having primary colors: red, 
green and blue (RGB); and filters with complimentary colors: cyan, yellow, green, magenta 
(Cy,Y,G,Mg) and white, cyan, yellow and green (W,Cy,Y,G). 

The correlated double sampling and automatic gain control device 14 conditions the 
analog signal output by the CCD 13 and passes the conditioned signal to A/D converter 1 5. The 
analog video signal is converted to a digital video signal by analog/digital converter 15 and the 
digital signal is passed to DSP 1 6. DSP 1 6 performs color transformation to, for example, the 
YUV, which is the predominant choice both in the digital television industry and for PC digital 
video and teleconferencing applications, or YIQ format and further manipulates image quality, as 
is known to those having ordinary skill in the art. In fact, traditional video cameras perform a 
large, amount of analog and digital signal processing. The hardware that implements this 
processing is responsible for a considerable amount of camera cost. 

Microcontroller 17 controls the operation of DSP 16 as understood by those having 
ordinary skill in the art. 



WO 97/24880 PCT/US96/2Q489 

-2- 

The YUV format nominally includes a total of 24 bits of data, 8 bits for each of the Y, U 
and V components. Typically, the YUV signal is subsampled, for example by DSP 16 prior to 
transmission to PC 1 1 to reduce the number of bits which must be transmitted through bus 19. 
Popular subsampled YUV formats include YUV 1 6, YUV 1 2 and YU V9. 

In YUV9, for example, pixels are grouped into 4x4 blocks, for a total of 1 6 pixels. Each 
of the 1 6 pixels maintains its separate 8 bit Y luminance information. Each pixel, however, 
shares a single 8 bit U and a single 8 bit V chrominance component. The total number of bits 
which must be sent to represent each pixel is 9: ((16 Y * 8 bits) + (1U * 8 bits) + (1 V * 8 
bits))/l 6 pixels = 144 bits/16 pixels= 9 bits per pixel (9bpp). YUV9 subsampling thereby 
reduces the data required to represent one pixel from 24 bits to 9 bits without a severe effect on 
image quality, since human viewers have poor acuity for color transitions in an image but 
relatively high acuity to intensity level changes. 

YUV 1 6 and YUV 1 2 color subsampling employ similar techniques on a smaller matrix of 

pixels. 

After converting digitized raw CCD color information to, for example, a YUV format and 
subsampling the data, the camera must transmit the video data to PC 11, for example for storage 
or display on an attached monitor (not shown in Fig. 1). The video data typically is transmitted 
to PC 11 through a parallel or serial port and over Universal Serial Bus (USB) 19. 
Unfortunately, even the most advanced USB has a theoretical bandwidth limit of 12Mbit/second, 
the maximum sustainable usable rate is more accurately about 6 Mbit/second. The bandwidth 
limitations imposed by the USB have a significant adverse effect on the quality of full motion PC 
video. For example, the following table shows maximum sustainable frame rates at various 
resolutions for YUV9 and YUV 12 data through a USB: 



Color\ 
Resolution 


YUV9 


YUV12 


160 * 120 


34.7 fps 


26.0 fps 


320 * 240 


8.7 fps 


6.5 fps 



Table 1 . Video Transmission Frame Rates for YUV Color Formats 

The minimum refresh rate for acceptable full motion video is at least 30 fps. As can be 
seen from the above table, a USB bus is capable of delivering only YUV9 subsampled video data 
at low resolution at the rate required for acceptable full-motion video. 
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A system and method are needed, therefore, for transmitting digital video signals from a 
digital video camera at a rate sufficient to reproduce full motion video over transmission media 
having bandwidth constraints. 

SUMMARY OF THE INVENTION 
An image is captured by a camera which has a sensor for generating a plurality of analog 
video signals in response to the image. The analog video signals are converted into a plurality of 
raw digital video signals defining the image. The raw digital video signals are transmitted to a 
computer over a bus. In an embodiment, the resolution of the image defined by the raw digital 
video signals is decreased prior to being transmitted over the bus. 

BRIEF DESCRIPTION OF THE DRAWINGS 
The following detailed description will be more fully understood with reference to the 
accompanying drawings in which: 

Fig. 1 is a block diagram of a prior art digital camera to PC via bus coupling; 

Fig. 2 is a block diagram of the digital camera to PC via bus coupling of the invention; 

Fig. 3a is a horizontal element of a downsampling filter; 

Fig. 3b is a vertical element of a downsampling filter; 

Fig. 4 is a block diagram of an output, of a CCD sensor in a primary color layout; 
Fig. 5a is another embodiment of a horizontal element of a downsampling filter; 
Fig. 5b is another embodiment of a vertical element of a downsampling filter; and 
Fig. 6 is a block diagram of an output of a CCD sensor in a secondary color layout. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 
Referring to Fig. 2, an embodiment of the invention is shown. As is evident from Fig. 2, 
digital video camera 20 does not include any DSP hardware. Any necessary transformation of 
raw digitized CCD video data to NTSC (YIQ) or PAL (YUV) format preferably is done using 
appropriate software running on PC 28. In an embodiment PC 28 further performs additional 
manipulations of the video data as discussed further below. 

Video camera 20 has lens 21 for focusing on a scene or the like denoted by optical path 
22. CCD sensor 24 generates an analog video signal which is passed to correlated double 
sampling and automatic gain control device 23 for conditioning. The conditioned raw CCD 
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analog video signal is converted into a raw CCD digital video signal by A/D converter 26. 
Camera 20 also preferably includes microcontroller 25. 

Color filters are preferably placed over CCD 24 to generate color analog video signals. 
As noted above, the color filters may be in primary or secondary color layouts. When using a 
5 primary color layout, the sensor output typically produces alternating lines of red, green, red, 
green, red, green . . . (R,G,R,G,R,G...) and green, blue, green, blue, green, blue . . . 
(G,B,G,B>G,B...) colors represented as analog color video signals. The raw analog color signals 
from CCD 24 are converted into raw digital signals by analog-to-digital (A/D) converter 26. 

In one complimentary color layout, CCD 23 typically produces alternating lines of G+Cy, 

1 0 Mg+ Ye, G+Cy, Mg+ Ye, G+Cy, Mg+Ye ... and Mg+Cy, G+Ye, Mg+Cy , G+Ye, Mg+Cy, G+ Ye 
. . . color video signals. The analog color video signals are digitized by A/D converter 26. The 
raw digital CCD color video signals in a secondary color layout are further processed such as to 
convert them into the RGB color space. In one embodiment, camera 20 includes the necessary 
processing hardware and/or software required for converting the secondary colors into an RGB 

1 5 format of video data. In another embodiment, PC 28 converts the raw digital CCD video data in 
complimentary color layout to RGB data, for example under software control. 

For example, in the latter embodiment, digital camera 20 is coupled through a bus having 
limited bandwidth, for example USB bus 27 to PC 28. Digitized raw CCD sensor video data 
from A/D converter 26 is transmitted over bus 27 for processing prior to display or storage. For 

20 example, PC 28 preferably generates raw luminance and color difference signals from the raw 
digital CCD signals in the complimentary color layout. PC 28 converts the color difference 
signals to RGB values which then can be transformed into a composite video format, such as 
NTSC (YIQ) or PAL (YUV). Additional processing may be performed by PC 28 on, for 
example, the RGB color signals, such as white balancing and gamma correcting. The video data 

25 may be displayed, for example on a video monitor associated with PC 28 (not shown in Fig. 2). 
For a further disclosure of the conversion of raw digital CCD color signals in a complimentary 
color layout to RGB signals, and processing of RGB color signals, see commonly owned U.S. 
Application No. 08/520,61 1, filed on August 30, 1995, incorporated herein by reference. 

The raw digitized CCD video data transmitted over bus 27, whether in a primary or 

30 secondary color layout, has 8 bits of color information per pixel. The 8 bit size of each raw 

digitized CCD color signal is of course less than video data transmitted in, for example, YUV9, 
YUV 12 or YUV 16 formats, which effectively have 9, 12 and 16 bits per pixel, respectively. 
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However, even though the raw digitized CCD video information has fewer bits per pixel than 
any of the YUV formats, it has more color information in those 8 bits than any of the YUV 
formats, which have a single pair of 8 bit color difference numbers, U and V, shared by 16, 4 and 
2 pixels, respectively. It can be seen, therefore, that the conversion of the RGB color signal into 
a YUV or YIQ format by DSP 16 in conventional systems as shown in Fig. 1 inefficiently 
generates redundant information which nevertheless must be transmitted over bus 19, thereby 
occupying valuable bus bandwidth. 

In a preferred embodiment shown in Figs. 3a and 3b, the digital video signals which 
represent raw CCD digital video signals are subjected to low-pass or downsampling filtering by 
filter 30 to decrease the resolution of the video image being transmitted over bus 27. 
Downsampling as herein described further decreases the bit-rate and enhances an ability to use a 
limited bandwidth bus 27 for transmission of video data to, for example, a PC at a sufficiently 
high refresh rate. 

The horizontal position 3 1 of a combination downsampling filter 30 is shown in Fig. 3a. 
A/D converter 26 of camera 20 having primary color layout produces alternating lines of 
RGRGRG and GBGBGB digital video data, representing raw sensor output as shown, for 
example, in Fig. 4. The digital video data is horizontally downsampled by horizontal 
downsampling filter 3 1 , which decreases by approximately 33% the horizontal resolution of the 
video image. For example pixel element 0,0 is multiplied by 0.5 (or divided by 2) by multiplier 
32.1 and the result is passed to adder 33.1. Similarly, pixel element 0,2 also is multiplied by 0.5 
(or divided by 2) by multiplier 32.3 and the resultant value is transmitted to adder 33.1 . 
Multipliers 32.1 and 32.3 may be hardware devices, for example a shifter, for shifting right the 
digital value of pixel elements 0,0 and 0,2. In another embodiment, the multiplication or 
division may be implemented in software and under processor control. Pixel element 0,1, which 
is a green pixel element is multiplied by multiplier 32. 1 having a coefficient of 0 so that the red 
pixel elements are separately downsampled. 

Adder 33.1 combines the values from multipliers 32. K 32.2 and 32.3 to provide a single 
digital value 34 representing downsampled pixel elements 0,0 and 0,2. 

Similar downsampling filtering is performed on groups of pixel elements 0,2 - 0,4; 0,4 - 
0,6; 0 ? 6 - 0,8 . . . for the entire first horizontal row 0 as shown in Fig. 3a. The entire filter matrix 
is then shifted a single pixel element to the right and the green pixel elements of the first 
horizontal row are downsampled. For example, pixel element 0,1 is multiplied by 0,5, pixel 
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element 0,2 is multiplied by 0 and pixel element 0,3 is multiplied by 0.5. The results of the 
multiplications (or divisions as described above) are added to provide, for example, a single 
digital value representing downsampled pixel elements 0,1 and 0,3. Downsampling filtering is 
correspondingly performed on pixel element groups 0,3 - 0,5; 0,5 - 0,7; 0,7 - 0,9... for the 
5 entirety of row 1 . 

After downsampling horizontal row 0, rows 1 - n are downsampled in an identical 
fashion. 

After performing horizontal downsampling, a preferred embodiment of the invention 
vertically downsamples using a vertical downsampling filter, such as vertical downsampling 

10 filter 39 of downsampling filter 30, for example to decrease by approximately 33% the vertical 
resolution of the image to be transmitted over bus 27. An embodiment of a vertical 
downsampling filter 39 is shown in Fig. 3b. Pixel elements 0,0, 1 ,0, and 2,0 are multiplied by 
multipliers 35.1, 35.2 and 35.3, respectively, having coefficients of 0.5, 0 and 0.5, and the 
resulting values are added by adder 36.1 to provide a single digital value 37. As discussed 

1 5 above, an alternative to multiplying by 0.5 is to divide by 2, a mathematical operation easily 
accomplished in digital electronics by shifting right. 

The value from adder 36.1 is the downsampled value of pixel elements 0,0 - 2,0 and more 
particularly the red pixel elements 0,0 and 2,0. After downsampling all of the red pixel elements 
in column 0, the entire filter matrix is shifted down one pixel element and downsampling is run 

20 on pixel groups 1,0 - 3,0; 3,0 - 5,0; 5,0 - 7,0; 7,0 - 9,0, which has the effect of isolating and 
downsampling the green pixel elements in column 0. After down sampling column 0, 
downsampling is performed on columns 1 - n in the manner described above. 

Downsampling filters of varying configurations are possible, as is evident to a person of 
ordinary skill relying on this disclosure, to decrease the resolution of an image output by digital 

25 camera 20 by varying percentages. For example, Figs. 5a and 5b show downsampling filter 50 

having horizontal and vertical elements 51 and 59 respectively, for further reducing, as compared 
to the filter of Fig. 3, the resolution of an image in two dimensions. Downsampling filter 
elements 5 1 and 59 have multipliers with coefficients of 1/3 and 0 for filtering colors 
independently, in the manner described above with reference to Figs 3a and 3b. A decrease in 

30 resolution typically enables an increase in a frame refresh rate. 

A digital camera 20 having a secondary color layout will generate the color pattern shown 
in Fig. 6. In a preferred embodiment of the invention, downsampling is performed on a 
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complimentary color layout pattern in the same manner as described above with reference to a 
digital video camera having a primary color layout. Any necessary conversion from a secondary 
color layout to, for example, a linear RGB format and an NTSC or PAL format preferably occurs 
in PC 28 after the downsampled, raw, digitized secondary color video signals are transmitted 
over bus 27. Conversion to RGB can be done under software control as will be understood by 
those skilled in the art, particularly when referring to Application No. 08/520,61 1 , filed on 
August 30, 1995, and incorporated herein by reference. 

The invention takes advantage of the smaller number of bits per pixel of video data 
associated with the digitized, raw output from a CCD of a digital camera as compared to the bits 
per pixel of an identical signal which has been converted to a YUV or YIQ format. The 
invention enables transmission of digital video data from a digital camera to a PC through a bus 
having limited bandwidth by moving any YUV or YIQ conversion steps out of the camera and 
into a PC, where they can be done by a high-speed PC microprocessor efficiently and without 
occupying many processor cycles. Advantageously, any other video data manipulations which 
are required also can be performed by the PC. For example, software running on an INTEL 
PENTIUM processor operating at 100 MHz was able to do the work needed to convert digitized 
CCD sensor signals into high quality video signals suitable for video conferencing and other 
video processing applications. At typical video conferencing resolution and frame rate, the CPU 
workload was less than 5% of the 100 MHz PENTIUM cycles. By eliminating a need for 
sophisticated processing hardware in a digital video camera, the invention provides the additional 
advantage of enabling the manufacture of less expensive digital video cameras. 

It should be understood that various changes in the details, materials, and arrangements 
of the parts which have been described and illustrated in order to explain the nature of this 
invention may be made by those skilled in the art without departing from the principle and 
scope of the invention as expressed in the following claims. 
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CLA I MS 

What we claim is: 

1 . A digital video camera, comprising: 

(a) means for generating a plurality of raw analog video signals in response to an image; 
5 (b) means for converting the raw analog video signals into a plurality of raw digital video 

signals defining the image; and 

(c) means for transmitting the raw digital video signals to a computer over a bus. 

2. The camera of claim 1 , further comprising: . 

10 (d) means for decreasing the resolution of the image defined by the raw digital video signals. 

3. The camera of claim 2, wherein means (a) comprises a charge coupled device. 

4. The camera of claim 3, wherein means (a) further comprises a color filter. 

15 

5. The camera of claim 4. wherein the color filter includes one of the sets of colors of red, 
green and blue, and magenta, yellow, cyan, and green. 

6. The camera of claim 2, wherein means (b) comprises at least one analog-to-digital 
20 converter. 

7. The camera of claim 2, wherein the bus has a bandwidth of not greater than 1 2 Mbits 
per second. 

25 8. The camera of claim 2, wherein means (d) comprises a downsampling filter. 

9. The camera of claim 8. wherein means (d) further comprises: 

(i) a horizontal downsampling filter to enable a reduction of resolution in a horizontal 
dimension; and 

30 (ii) a vertical downsampling filter to enable a reduction of resolution in a vertical 

dimension, and wherein each of the horizontal and vertical downsampling filters includes 
coefficients of 0. 
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10. The camera of claim 2, wherein the computer has processor means for converting the 
plurality of raw digital video signals into a plurality of usable digital video signals. 

1 1 . The camera of claim 1 0, wherein the usable digital video signals are one of YUV and 
YIQ signals. 

12. A method for generating digital video signals using a digital video camera, 
comprising the steps of: 

(a) generating a plurality of raw analog video signals in response to an image; 

(b) converting the raw analog video signals into a plurality of raw digital video signals 
defining the image; and 

(c) transmitting the raw digital video signals to a computer over a bus. 

13. The method of claim 1 2, further comprising the step of 

(d) decreasing the resolution of the image defined by the raw digital video signals. 

14. The method of claim 1 3, wherein step (a) comprises the step of focusing the image 
onto a charge coupled device. 

15. The method of claim 14, wherein the charge coupled device is overlaid with one or 
more color filters. 

16. The method of claim 1 5 wherein the one or more color filters include one of the sets 
of colors of red, green and blue, and magenta, yellow, cyan, and green. 

1 7. The method of claim 13, wherein step (b) comprises the step of processing with at 
least one analog-to-digital converter. 

1 8. The method of claim 1 3, wherein the bus has a bandwidth of not greater than 12 Mbits 
per second. 



1 9. The method of claim 13, wherein step (d) comprises the step of downsample filtering. 
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20. The method of claim 19, wherein step (d) further comprises the steps of: 

(i) horizontal downsample filtering to enable a reduction of resolution in a horizontal 
dimension; and 

(ii) vertical downsample filtering to enable a reduction of resolution in a vertical 
5 dimension, and wherein each of the horizontal and vertical downsample filtering uses 

coefficients of 0. 

2 1 . The method of claim 1 3, wherein the computer comprises a processor for converting 
the plurality of raw digital video signals into a plurality of usable digital video signals. 

10 

22. The method of claim 21, wherein the usable digital video signals are one of YUV and 
Y1Q signals. 

23. A device for processing video signals, comprising: 

15 (a) means for receiving over a bus a plurality of raw digital video signals defining an image; 

and 

(b) means for converting the plurality of raw digital video signals into a plurality of usable 
digital video signals, wherein the raw digital video signals are a digitized version of raw charge 
coupled device output signals. 

20 

24. The device of claim 23, wherein the plurality of raw digital video signals have been 
generated by a digital video camera equipped with means for decreasing the resolution of the 
image defined by the raw digital video signals. 

25 25. The device of claim 23, wherein the bus has a bandwidth of riot greater than 12 

Mbits/second. 

26. The device of claim 23, wherein the usable video signals are one of YUV and YIQ 

signals. 
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27. The device of claim 23, wherein the digital video camera has an analog to digital 
converter and the raw digital video signals generated by the camera are a raw analog output of 
the charge coupled device which has been processed by the analog to digital converter. 



28. The device of claim 23, wherein means (b) comprises a computer processor. 
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